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Abstract 

Compressed sensing is a technique for finding sparse solutions to un- 
>«✓ , derdetermined linear systems. This technique relies on properties of the 

Q^) ■ sensing matrix such as the restricted isometry property. Sensing matrices 

, that satisfy this property with optimal parameters are mainly obtained 

via probabilistic arguments. Deciding whether a given matrix satisfies the 
restricted isometry property is a non-trivial computational problem. In- 
deed, we show in this paper that restricted isometry parameters cannot 
be approximated in polynomial time within any constant factor under the 
assumption that the hidden clique problem is hard. 

Moreover, on the positive side we propose an improvement on the 
■ brute-force enumeration algorithm for checking the restricted isometry 

property. 
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1 Introduction 



Let $ be a n x N matrix with N > n. A vector x G is said to be k- 
sparse if it has at most k nonzero coordinates. Given 5 G]0, 1[, (f> is said to 
satisfy the Restricted Isometry Property (RIP) of order k with parameter 5 if 
it approximately preserves the Euclidean norm in the following sense: for every 
fc-sparse vector x, we have 

{l-5)\\x\\ 2 < \\$x\\ 2 < (1 + <5)||x|| 2 . 

Clearly, for this to be possible we must have k < n. Given 5, n and N, the 
goal is to construct RIP matrices with k as large as possible. This problem is 
motivated by its applications to compressed sensing: it is known from Candes, 
Romberg and Tao QUI [11] that the restricted isometry property enables the 
efficient recovery of sparse signals using linear programming techniques. For 
that purpose one can take any fixed 5 < v2 — 1 [9] . 
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Various probabilistic models are known to generate random matrices that 
satisfy the RIP with a value of k which is (almost) linear n. See for instance 
Theorem 2 in Section [3] for the case of matrices with entries that are independent 
symmetric (±1) Bernouilli matrices. The recent survey [22] provides additional 
results of this type and extensive references to the probabilistic literature. Some 
significant effort has been devoted to the construction of explicit (rather than 
probabilistic) RIP matrices, but this appears to be a difficult problem. As 
pointed out by Bourgain et al. in a recent paper 0[8], most of the known explicit 
constructions [TBI El CH| are based on the construction of systems of unit vectors 
with a small coherence parameter (see section[2]for a definition of this parameter 
and its connection to the RIP). Unfortunately, this method cannot produce RIP 
matrices of order k > y/n [7J [8]. Bourgain et al. still manage to break through 
the -y/ra "barrier" using techniques from additive combinatorics: they construct 
RIP matrices of order k = n l / 2+t ° where eo > is an unspecified "explicit 
constant". Note that this is still far from the order achieved by probabilistic 
constructions. 

Here we study the restricted isometry property from the point of view of 
computational complexity: what is the complexity of deciding whether a matrix 
satisfies the RIP, and of computing or approximating its order k or its RIP 
parameter 51 An efficient (deterministic) algorithm would have applications to 
the construction of RIP matrices. One would draw a random matrix <3? from 
one of the well-established probabilistic models mentioned above, and run this 
hypothetical algorithm on <3? to compute or approximate k and 5. The result 
would be a matrix with certified restricted isometry properties (see Section [3] for 
an actual result along those lines) . This may be the next best thing short of an 
explicit construction (and as mentioned above, the known explicit constructions 
are far from optimal). 

The definition of the restricted isometry property suggests an exhaustive 
search over ( fc ) subspaces, but prior to this work there was little evidence that 
checking the RIP is computationally hard (more on this in Section \1.2\i . There 
has been more work from the algorithm design side. In particular, it was shown 
that semi-definite programming can be used to verify the restricted isometry 
property [12J and other related properties from compressed sensing [HI IT?] . 
Unfortunately, as pointed out in [13] these methods are unable to certify the 
restricted isometry property for k larger than 0(^/n), even for matrices that 
satisfy the RIP up to order Q(n). As we have seen, k = 0{y/n) is also the range 
where coherence-based methods reach their limits. 

In this paper we provide both positive and negative results on the compu- 
tational complexity of the RIP, including the range k > y/n. 

1.1 Positive Results 

In Section [2j we study the relation between the RIP parameters of different 
orders for a given matrix $. Very roughly, we show in Theorem [TJ that the RIP 
parameter is at most proportional to the order. We therefore have a trade-off 
between order and RIP parameter: in order to construct a matrix of given order 
and RIP parameter, it suffices to construct a matrix of lower order and smaller 
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RIP parameter. We illustrate this point in Section [3j Our starting point is the 
above-mentioned (very naive) exhaustive search algorithm, which enumerates 
all (^) subspaces generated by k column vectors. We obtain a "lazy algorithm" 
which enumerates instead all subspaces generated by I basis vectors for some 
/ < k. We show that the lazy algorithm can go slightly beyond the -y/ra barrier 
if a quasi-polynomial running time is allowed. 

1.2 Negative Results: the Connection to Hidden Cliques 

We show that RIP parameters are hard to approximate within any constant 
factor under the assumption that the hidden clique problem is hard. In fact, 
we need an assumption (spelled out at the end of this subsection) which is 
somewhat weaker than the usual one. Our hardness result applies to any order 
of the form k = n a , where a is any constant in the interval ]0, 1[. It applies to 
square as well as to rectangular matrices. We gave similar results in the unpub- 
lished manuscript [19] under a (nonstandard) assumption on the complexity of 
detecting dense subgraphs. By contrast, as explained below the hypothesis that 
we use in this paper is well established. Prior to our work, little was known 
on the hardness of checking the restricted isometry property. It was pointed 
out by Terence Tao [21] that "there is no fast (e.g. sub-exponential time) algo- 
rithm known to test whether any given matrix is UUP or not."0 As to hardness 
results, one can mention the NP-hardness proof of [6], which is based on the 
following (known) fact: it is NP-hard to distinguish a matrix with a nonzero 
fc-sparse vector in its kernel from a matrix without any such vector in its kernel. 
In the first case, the matrix does not satisfy the RIP of order k, while in the 
second case it does satisfy the RIP of order k for some parameter 5. Since 8 
may be very close to 1, this result does not say much on the complexity of 
approximating the RIP parameters. A similar result was obtained in [20J. 

The size of the largest clique in a typical graph drawn from the G(n, 1/2) 
distribution is roughly 21og 2 n. In the hidden clique problem, one must find 
a clique of size t S> 2 log 2 n which was planted at random in a random graph. 
This problem is solvable in polynomial time for a clique of size t = Q(^/n) [5]. 
It is widely believed, however, that the problem cannot be solved in polynomial 
time for a planted clique of size t = n c , where c is any constant in the open 
interval ]0, 1 /2[. Even the more modest goal of distinguishing between a random 
graph and a random graph with a planted clique of size n c is believed to require 
more than polynomial time [2] (see appendix B.4 of [1] for a comparison of 
distinguishing versus finding hidden cliques). 

In the last few years, several hardness results have been obtained under the 
assumption that the hidden clique problem is not polynomial time solvable [2j 
[TJ[TB]. We refer to [2] for more information on the history of this problem. 

In this paper, we show hardness of approximation for RIP parameters under 
the following assumption. We actually have a family of assumptions, parame- 
terized by the clique size (in keeping with the tradition in this area [1] , we omit 
floor and ceiling signs to simplify the presentation). 

1 In his blog post, Tao uses the notation "UUP" for the RIP. 
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Hypothesis (H e ). There is no polynomial time algorithm A which, given as 
input a graph G on n vertices: 

• always outputs "yes" if G contains a clique of size n~^ e . 

• Outputs "no clique " on most graphs G when G is drawn from the uniform 
distribution G(n, 1/2). 

In other words, (H e ) asserts that no polynomial time algorithm can certify 
the absence of a clique of size n2" f from most graphs on n vertices (where 
"most graphs" means: with probability approaching 1 as n — > +00). Note that 
this is a one-sided hypothesis: algorithm A is allowed to err (rarely) but only 
on input graphs that do not contain a a clique of size n2~ e . 

Note also that Hypothesis H e becomes increasingly stronger as e — > (and it 
becomes false for e < 0: if a > 1/2, a simple spectral algorithm can certify that 
most graphs on n vertices do not contain any clique of size n a . For completeness, 
we give a proof in the appendix). Hypothesis H e is clearly true if it is hard to 
distinguish beween a random graph and a random graph with a planted clique 
of size n2~ e . It is therefore consistent with current knowledge to assume that 
(H e ) holds true for all constants e G]0, l/2[. 

1.3 Organization of the Paper 

As explained above, the next two sections are devoted to positive results. In 
Section |4] we work out some bounds on the eigenvalues of random matrices, 
for later use in our reductions from hidden clique to the approximation of RIP 
parameters. We rely mainly on the classical work of Fiiredi and Komlos [15] as 
well as on a more recent concentration inequality due to Alon, Krivelevich and 
Vu [5]. In Section [5] we use these eigenvalue bound to show that approximat- 
ing RIP parameters is hard even for square matrices. In Section [6] we derive 
similar results for matrices of "strictly rectangular" format (which is the case 
of interest in compressed sensing). We proceed by reduction from the square 
case. Interestingly, this last reduction relies on the known constructions (de- 
terministic [TIE] and probabilistic [22]) of matrices with good RIP parameters 
mentioned earlier in the introduction. We therefore turn these positive results 
into negative results. The table at the end of Section [6] gives a summary of our 
hardness results. 

2 Increasing the Order by Decreasing the RIP Pa- 
rameter 

As explained at the beginning of 0[B], certain (suboptimal) constructions are 
based on the construction of systems of unit vectors (u\, . . . , wjv) G C n with 
small coherence. The coherence parameter \x is defined as max^ | (m , Uj ) \ . 
Indeed, we have the following proposition. 

Proposition 1. Assume that the column vectors u±, . . . , un of are of norm 1 
and coherence \x. Then $ satisfies the RIP of order k with parameter 5 = 
(k-l)fi. 
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We reproduce the proof from [TJ [8] since if fits in one line: for any ^-sparse 
vector x, 

|||$x|| 2 - ||x|| 2 | < 2y^ j \x i Xj(uj,Uj)\ < |ti((y~] \xi\) 2 - \\x\\ 2 ) < (k - l)n\\x\\ 2 . 

i<j i 

We now give a result, which (as we shall see) generalizes Proposition [TJ 

Theorem 1. Assume that <3? has unit column vectors and satisfies the RIP of 
order m with parameter e. For k > m, <3? also satisfies the RIP of order k with 
parameter 5 = e(k — l)/(m — 1). 

Proof. Let u\, . . . , un be the column vectors of <£. Let x be a £;-sparse vector, 
and write x = Y2ieT XiUi wnere 7" is a subset of {1, ... ,N} of size k. Since 
||$x|| 2 = ||x|| 2 + 2^2 i<c j XiXj(ui,Uj), to check the RIP of order k we need to 
show that 

| ^2xiXj(ui,Uj}\ < 5||x|| 2 /2, (1) 

i<j 

where 5 = e(k — l)/(m — 1). To estimate the left hand side, we compare it to 
the sum of the similar quantity taken over all subsets of size m of T, namely: 

I Yl Yl XiXj(ui,Uj)\. (2) 

\S\=mi,j£S,i<j 

Since each pair (i,j) appears in exactly (^"2) subsets of size m, this sum is 
equal to (^"2) times the left-hand side of (JJ. But we can also estimate ([2]) 
using the RIP of order m. For each subset S of size m, we have 



XiXj{ui,Uj)\ < e^x 2 /2. 



E 

i,j£S,i<j i&S 



This follows from ([TJ), replacing 5 by e (the RIP parameter of order m). Since 
each term xf will appear exactly in subsets, we obtain e( ) ^~ 1 1 ) | |x| | 2 /2 as 

an upper bound for ([2]). We conclude that the left-hand side of ([lj is bounded 

byf(r_ 1 1 )lkll 2 /(r_ 2 2 )=^INI 2 /2. □ 

We claim that Proposition [TJ is the case m = 2 of Theorem [TJ This follows 
from the following observation. 

Remark 1. For a matrix with unit column vectors, the coherence parameter 
\x is equal to the RIP parameter of order 2. 

Proof. Let 5 be the RIP parameter of order 2. We have 5 < by Proposition [TJ 
It remains to show that 5 > fi. Consider therefore two column vectors Ui and 
Uj with I (ui, Uj) I = 11. Let x = Ui + uj. We have ||x|| 2 = 2 and ||^x|| 2 = 2±2^, 
so that 5 > /i indeed. □ 



5 



3 A Matrix Certification Algorithm 



The naive algorithm for computing the RIP parameter of order k will involve 
the enumeration of the (^) submatrices of <3? made up of k column vectors of 
<£. For each T C {1, . . . ,N} of size k let us denote by &t the corresponding 
nx k matrix. We need to compute (or upper bound) 5 = maxT^T, where 

5 T = sup | ||$tz|| 2 /IM| 2 - 1 |. 

xec k 

For each T, 5t can be computed efficiently by linear algebra. For instance, 5t 
is the spectral radius of the self-adjoint matrix & t &t — Ifc- The cost of the 
computation is therefore dominated by the combinatorial factor (^) due to the 
enumeration of all subsets of size k. 

Here we analyze what the naive algorithm can gain from Theorem [TJ We 
therefore consider the following lazy algorithm. The correctness of the algorithm 

Algorithm 1 
1: procedure Lazy($, m, 5) 

2: Input: a n x N matrix $ with unit column vectors, an integer m < n, 

and a parameter 5 G]0, 1[. 
3: Compute as explained above the RIP parameter of order m. Call it e. 
4: Output: Certify $ as a RIP matrix of order k with parameter 8, for 

all k > m such that e(k — l)/(m — 1) < 5. 
5: end procedure 



follows immediately from Theorem [H We now analyze its behavior on random 
matrices, which are in many cases known to satisfy the RIP with high proba- 
bility. Consider for instance the case of a matrix whose entries are independent 
symmetric Bernouilli random variables. 

Theorem 2. Let A be anxN matrix whose entries are independent symmetric 
Bernouilli random variables and assume that n > Ce -2 mlog(eiV/m). With 
probability at least 1 — 2exp(— ce 2 n), the normalized matrix $ = satisfies 
the RIP of order m with parameter e. Here C and c are absolute constants. 

In fact the same theorem holds for a very large class of random matrix mod- 
els, namely, subgaussian matrices with either independent rows or independent 
columns (|22j. Theorem 64). 

Proposition 2. Let A be a random matrix as in Theorem^ and 5 g]0, 1[. 
With probability at least 1 — 2(eN/m)~ cCm , the lazy algorithm presented above 
will certify that A satisfies the RIP of order k with parameter 5 for all k such 
that: 

k <5 I mn 

Y c\og{eN/m) 

Here c and C are the absolute constants from Theorem 0- 

Proof. All parameters being fixed we take e as small as allowed by Theorem [21 
so that ne 2 = Cmlog(eN/m). This yields the announced probability estimate, 
and the upper bound on k is 5m/ e. □ 
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To compare the lazy algorithm to the naive algorithm, set for instance m = 
y/n. In applications to compressed sensing one can set 5 to a small constant 
value (any S < y/2 — 1 will do). Thus, disregarding constant and logarithmic 
factors, with high probability the lazy algorithm will certify the RIP property 
for k of order roughly n 3 / 4 . This is achieved by enumerating L1/2) subspaces, 
whereas the naive algorithm would enumerate roughly ( 3/4) subspaces. 

Another choice of parameters in Proposition [2] shows that one can beat the 
yfn bound by a logarithmic factor with a quasi-polynomial time algorithm. For 
instance: 

Corollary 1. If we set m = (log N) 3 , the lazy algorithm runs in time 2°( log N ^ 
and, with probability at least 1 — 2~^(( logAr ) ) certifies that A satisfies the RIP 
of order k with parameter 5 for all k < Kb log N\fn, where K is an absolute 
constant. 

4 Eigenvalues of Random Symmetric Matrices 

Proposition |4] is the main probabilistic inequality that we derive in this section. 
It shows that square matrices obtained by Cholesky decomposition from a cer- 
tain class of random matrices have good RIP parameters with high probability. 
This result is then used in Section [5] to give a reduction from hidden clique to 
the approximation of RIP parameters. 

4.1 Model A 

Consider the following random matrix model: A is a symmetric k x k matrix 
with an = 0, and for i < j the independent symmetric Bernouilli random 

variables. 

Let \\{A) > \2{A) > . . . Xf t (A) be the eigenvalues of A. Let m s be the 
median of A s (^4). From the main result of [5] (bottom of p. 263) we have for 
t > the inequality: 

Pt[Xs(A) -m s >t]< 2e~ t2/32s \ 

From Fiiredi and Komlos (|15j. Theorem 2) we know that mi < 3ayk for k 
large enough, where a = 1 is the standard deviation of the a™ in the case i < j. 
Therefore we have 

Pr[Ai(A) > 3Vk + t} < 2e- {2/32 . 
Since \k(A) = — X\(— A) and —A has same distribution as A, we also have 

Pr[A fc (A) < -3Vk-t] < 2e~* 2 / 32 

(one could also apply directly the bound on Xk(A) for the more general model 
considered in [5j). As a result: 

Proposition 3. There is an integer ko such that for all k > k^ and for all 

t > we have: 

Pr[max|Ai(A)| > 3^ + *] < 4e"* 2/32 . 
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Remark 2. The constant 3 in Proposition [3| can be replaced by any constant 
bigger than 2 (see Theorem 2 in [15]). 



4.2 Model B 

Next we consider the model where B is a symmetric k x k matrix satisfying 
the following condition: bu = 1, and bij = c ■ aijl^fn for i < j, where the 
are independent symmetric Bernouilli random variables. Here c > is a fixed 
constant, and n is an additional parameter which should be thought of as going 
to infinity with k. 

Corollary 2. Assume that k > ko and that 5^/n > ?>cyfk. Then the eigenvalues 
of B all lie in the interval [1 — 5,1 + 5] with probability at least 



l-4exp[-(^-3^) 2 /32]. 



Proof. We have 5 = 1/; + cA/ ^/n, where A follows the model of Proposition [3j 
The result therefore follows from that proposition by choosing t so that c{3>y/k + 
t)/ ' \fn = 5, i.e., t = 5^/njc — 2>Vk. □ 

In the next corollary we look at the case n = k of this model. 

Corollary 3. Assume that n > k§ and 3c < 1. Then B is positive semi-definite 
with probability at least 

l-4exp[-(l/c-3) 2 n/32]. 
Proof. Set n = k and 5 = 1 in Corollary [21 □ 

In the last result of this subsection we consider again the model B = I n + 
cA/yfn. Given a n x n matrix M and two subsets S,T C {1, . . . , n} of size k, 
let us denote by M$ t T the k x k sub-matrix made up of all entries of M of row 
number in S and column number in T. 

Corollary 4. Consider the random matrix B = I n + cA/^/n where A is drawn 
from the uniform distribution on the set n x n symmetric matrices with null 
diagonal entries and ±1 off- diagonal entries. 
If n> k > ko, then with probability at least 



1 — 4 exp 



k\n(ne/k) - - 3\/fc) 2 /32 



the submatrices Bs t s have all their eigenvalues in the interval [1 — 5,1 + 5] for 
all subsets S C {1, . . . , n} of size k. 

Proof. By Corollary [21 for each fixed S matrix Bs,s has an eigenvalue outside 

of the interval [1 — 5, 1 + 5] with probability at most 4exp[— — 3\/k) 2 /32]. 
The result follows by taking a union bound over the (^) < (ne/k) k subsets of 
size k. □ 
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4.3 Model C 



In Corollaries and SI we considered the following random model for B: set 
B = I n + cA/y/n, where A is chosen from the uniform distribution on the set 
S n of all symmetric matrices with null diagonal entries and ±1 off-diagonal 
entries. If B is positive semi-definite, we can find by Cholesky decomposition a 
n x n matrix C such that C T C = B. If B is not positive semi-definite, we set 
C = 0. This is the random model for C that we study in this subsection. 

Proposition 4. Assume that n > k > ko and that 3c < min(l, Sy/n/Vk). With 
probability at least 



1 — 4 exp 



k ln(ne/fe) 



[ 5 -^-3Vk) 2 /32 



4exp[-(l/c-3) 2 n/32], 



C satisfies the RIP of order k with parameter S. 



Proof. If B = I n + cA/y/n is not positive semi-definite then C = and this 
matrix obviously does not satisfy the RIP. By Corollary [21 B can fail to be 
positive semi-definite with probability at most 4 exp [-(l/c-3) 2 n/32]. 

If B is positive semi-definite then C T C = B. Using the notation of Corol- 
lary SI matrix C satisfies the RIP of order k with parameter 5 if for all subsets 
S of size k, the eigenvalues of the k x k matrices (C T C)s,s ah he in the interval 
[1 — 5, 1 + 5]. Since C T C = B, by Corollary S] this can happen with probability 
at most 4exp[Hn(ne//c) - - 3Vk) 2 /32}. □ 



5 Large Cliques and the Restricted Isometry Prop- 
erty 

In this section we show (in Theorems El SI and more generally in Theorem E]) 
that RIP parameters are hard to approximate even for square matrices. We 
establish connections between hidden clique problems and the RIP thanks to a 
generic reduction which we call the Cholesky reduction. This reduction maps a 
graph G on n vertices to a n x n matrix C(G). Let A be the signed adjacency 
matrix of G: we have an = and for i ^ j, aij = 1 if ij G E; aij = —1 if ij^E. 
We construct C = C(G) from A using the procedure described in Section l4~3l 
That is, we first compute B = I n + cA/y/n. Here c is some absolute constant 
smaller for 1/3, for instance c = 0.3. If B is not positive semi-definite, we set 
C = 0. Otherwise, we find by Cholesky decomposition a matrix C such that 
C T C = B. 

For suitable values of k, C(G) satisfies the RIP of order k for most graphs 
G. This was made precise in Proposition SI On the other hand, if G has a 
fe-clique then C(G) cannot satisfy the RIP of order k for a small value of the 
parameter 5. In order to show this, we first need a simple lemma. 

Lemma 1. Let A be the signed adjacency matrix of a graph G. If G has a 
clique of size k then there is a unit vector supported by k basis vectors such that 
x T Ax = k — 1. 
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Proof. Let H be the fc-clique. Here is a suitable vector: set x% = 1/Vk if i G 
and = otherwise. □ 



Proposition 5. 7f G /ias a clique of size k and 5 < c(k — l)/yjn then C(G) 
does not satisfy the RIP of order k with parameter 5. 

Proof. If B is not semi-definite positive, C(G) = does not satisfy the RIP. 
Otherwise C T C = B. Let x be the vector of Lemma [TJ We have ||Cx|| 2 = 
x T C T Cx = x T Bx = 1 + cx T Ax/^/n > 1 + 5. □ 

We can now prove our first hardness results. We first illustrate our method 
on two examples, and then prove a general result at the end of this section. 

Theorem 3. Assume hypothesis (H\/§), that is: no polynomial time algorithm 
can certify that most graphs do not contain a clique of size n 1 / 3 . Then, no 
polynomial time algorithm can distinguish a matrix with RIP parameter of order 
n 1//3 at most n" 1 / 4 from a matrix with RIP parameter of order n 1//3 at least 

Proof. We show the contrapositive: assuming the existing of a distinguishing 
algorithm A, we construct an algorithm that contradicts hypothesis (H 1 / 6 ). 
Fix a constant c < 1/3, for instance c = 0.3. On input G, this algorithm first 
contract C(G). 

If G contains a clique of size k = n 1//3 then by Proposition [5] the matrix 
C(G) does not satisfy the RIP of order k with parameter c'n -1 / 6 . Here d < c 
is another constant (for n large enough we can take d = 1/4). 

We consider now the case where G was drawn from the G(n, 1/2) distribu- 
tion. Set 5 = n" 1 / 4 . We can apply Proposition H] since 

3c. This proposition shows that with probability approaching 1 as n — > +oo, 
C(G) satisfies the RIP of order k with parameter S. 

We can therefore call algorithm A to certify the absence of a clique of 
size n 1 / 3 . More precisely, if G contains a fc-clique our algorithm always finds 
out. On the other hand, if G was drawn from G(n, 1/2) our algorithm answers 
correctly with high probability. □ 

This theorem implies in particular than RIP parameters cannot be approx- 
imated within any constant factor. We can obtain a similar result for an order 
k > y/n under the same hypothesis. This is possible essentially because a ma- 
trix that doesn't satisfy the RIP for a given order k cannot satisfy the RIP for 
any order k' > k. 

Theorem 4. Assume Hypothesis (Hi/q) as in the previous theorem. Then 
no polynomial time algorithm can distinguish a matrix with RIP parameter of 
order n ' 6 at most n~ ' 19 from a matrix with RIP parameter of order n 0,6 at 
least n _1 / 6 /4. 

Proof. We proceed as in the proof of the previous theorem: assuming the exist- 
ing of a distinguishing algorithm A, we construct an algorithm that contradicts 
the hypothesis. 
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If G contains a clique of size n 1//3 then we saw that C(G) does not satisfy 
the RIP of order n 1 / 3 with parameter n _1//6 /4. It is a fortiori the case that 
this matrix does not satisfy the RIP of order k = n 6 > n 1 / 3 with parameter 
n _1 / 6 /4. 

We consider now the case where G was drawn from the G(n, 1/2) distribu- 
tion. Set 5 = n -0 ' 19 . We can apply Proposition [J] since 

3c. Consider the argument of the first exponential in the probability bound 
of Proposition HI The positive term kln(ne/k), which is of order n 6 lnn, is 
dominated by the negative term (£^2 _ Sy/k) 2 /32, which is of order n ' 62 . We 
conclude that with probability approaching 1 as n — > +oo, C{G) satisfies the 
RIP of order k with parameter 5. 

We can therefore call algorithm A to certify the absence of a clique of 
size n 1 / 3 . More precisely, if G contains a clique of size n 1 / 3 our algorithm always 
finds out. On the other hand, if G was drawn from G(n, 1/2) our algorithm 
answers correctly with high probability. □ 

More generally, we have the following result. 
Theorem 5. Set k = nC 1 - 26 )^-*) where e e]0, l/2[. 5ei a/so 5 = n -5c/4+e 2 /2^ 

Hypothesis (H t ) implies that no polynomial time algorithm can distinguish 
a matrix with RIP parameter of order k at most 5 from a matrix with RIP 
parameter of order k at least n~ e /4. 

In particular, since 5 = o(ra~ e /4), it follows that no polynomial time al- 
gorithm can approximate the RIP parameter of order k within any constant 
factor. 

Remark 3. The exponent a = (1 — 2e)(l — e) ranges over ]0, 1[ as e ranges 
over the interval ]0, l/2[. This theorem therefore shows that for any exponent 
a G]0, 1[, the RIP parameter of order k = n a cannot be approximated within 
any constant factor in polynomial time. 

Proof of Theorem^ That 5 = o(n _e /4) follows from the inequality — 5e/4 + 
e 2 /2 < — e/4. This inequality holds true for all e G]0, 2[, and in particular for 
all e in the range ]0, l/2[ that is of interest here. 

We now prove the main part of the theorem. Assuming the existence of 
a distinguishing algorithm A, we construct again an algorithm that refutes 
hypothesis (H e ). 

We set as usual c = 0.3. If G contains a clique of size n 1 / 2-6 then by 
Proposition [5] C(G) does not satisfy the RIP of order n 1//2_€ with parameter 
n _e /4. It is a fortiori the case that this matrix does not satisfy the RIP of 
order k = n^ 2 "^ 1- ^ > n^ 2t ^ 2 for the same parameter value. 

Consider now the case where G is drawn from the G(n, 1 /2) distribution. 
We can apply Proposition [4] since b^fnj\fk = n*' 1-2 '' > 1 > 3c. Consider the 
argument of the first exponential term in the probability bound of Proposition^ 
The positive term kln(ne/k), which is of order fclnn = n <yl ^ 2e ^ 1 ^ t ^ Inn, is 
dominated by the negative term — 3\/fc) 2 /32, which is of order 5 2 n = 

n i-5e/2+e 2 _ j^gg^ ^g difference in the two exponents is 

l-| + e 2 -(l- 2e )(l- e ) = |- e 2 >0. 
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As a result, with probability approaching 1 as n — > +00, C(G) satisfies the 
RIP of order fc with parameter 6. We can therefore refute hypothesis (H e ) by 
running algorithm A on input C(G). □ 



6 Hardness for Rectangular Matrices 

In this section we show that the RIP parameters of rectangular matrices are 
hard to approximate. This is the case of interest in compressed sensing. In a 
sense this was already done in Section [5j we have shown that the special case 
of square matrices is already hard. Nevertheless, it is of interest to know that 
the problems remains hard for strictly rectangular matrices. This is what we 
do in this section. Proofs are essentially by reduction from the square case. We 
begin with a simple lemma. 

Lemma 2. Consider a matrix with the block structure 



where A and B both have at least k columns. This matrix satisfies the RIP of 
order k with parameter 5 if and only if the same is true for both A and B. 

Proof. For an input vector x with the corresponding block structure x = (u v) 
we have ||x|| 2 = ||u 2 || + \\v\\ 2 and ||^x|| 2 = ||^|| 2 + ||-£MP- Therefore, if $ 
satisfies the RIP of order k with parameter 5 then the same is true for A (take 
v = and u /c-sparse). The same argument applies also to B. 

Conversely, assume that A and B satisfy the RIP of order k with parameter 
6. Let x = (u v) be a /c-sparse vector. We have ||<5a;|p — \\x\\ 2 = (\\$>u\\ 2 — 
IMP) + (H^lP — IMP)- Both u and v must be A:-sparse, so the first term is 
bounded in absolute value by 5||u|| 2 and the second one by <5|M| 2 . The result 
follows since ||u|| 2 + |M| 2 = INP- □ 

Theorem 6. There are absolute constants eo,e > such that under hypothe- 
sis (H e ) and the choice of parameters: 



no polynomial time algorithm can distinguish a matrix with RIP parameter of 
order k at most 5 from a matrix with RIP parameter of order k at least n~ e /4. 

Moreover, polynomial-time distinction between these two cases remains im- 
possible even for matrices of size 2n x (n + N) where N = n 1+e ° . As a result, 
for matrices of this size the RIP parameter of order k cannot be approximated 
in polynomial time within any constant factor. 

The first part of the theorem follows from Theorem The point of Theo- 
rem [6] is that it establishes hardness of approximation for strictly rectangular 
matrices. 




= n 



5e/4+e 2 /2 
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Proof of Theorem The claim on constant factor approximation follows as in 
Theorem [5] from the relation 5 = o(n~ e /4). To prove the remainder of the 
theorem, we build on the proof of Theorem From a graph G on n vertices 
we construct the matrix 

where C (G) is as in the previous section and B n is a matrix with good restricted 
isometry properties. Its role is to ensure the rectangular format that we need 
for C'(G). Our specific choice for B n is the matrix constructed in [HE]. It is 
of size n x N where N = n 1+e °, and it satisfies the RIP of order nz +€ ° with 
parameter n~ e °. Moreover, B n can be constructed deterministically in time 
polynomial in n. Note that C'{G) is of size 2n x (n + N) as required in the 
statement of Theorem [6J 

Choose e so small that (1 - 2e)(l - e) > \ + e and -5e/4 + e 2 /2 > -e Q . 
We thus have 5 > n~ € °. It then follows from Lemma [2] that C'{G) satisfies the 
RIP of order k with parameter 5 if and only if C(G) does. 

To complete the proof, let us assume that we have a distinguishing algo- 
rithm A which works for matrices of size 2n x (n + N). We use it to refute 
hypothesis (H e ). 

If G contains a clique of size n 1//2_e , we saw in the proof of Theorem [5] 
that C(G) does not satisfy the RIP of order n l l 2 ~ e with parameter n _e /4 (by 
Proposition [5]). It is a fortiori the case that this matrix does not satisfy the 
RIP of order k = n^ +eo for the same parameter value, and the same is true of 
C'(G). 

Consider now the case where G is drawn from the G(n, 1/2) distribution. 
We saw in the proof of Theorem [S] that for most G, C(G) satisfies the RIP of 
order n^ 1_2e ^ 1_<E ^ with parameter 5. That order is at least as large as k = n2 +e °, 
so it is a fortiori the case that C (G) satisfies the RIP of order k with parameter 
5 for most G. As pointed out above, the same is then true for C'(G). We can 
therefore refute hypothesis (H e ) by running algorithm A on C'(G). □ 

Theorem [6] establishes hardness of approximation for an order k which is 
only slightly above n 1 / 2 . We can bring k much closer to n, but for this we need 
a randomized version of hypothesis {H e )\ 

Hypothesis (H^). There is no polynomial time randomized algorithm which, 
given as input a graph G on n vertices: 

• always outputs "yes" if G contains a clique of size n?~ e . 

• Outputs "no" with probability at least (say) 3/4 on most graphs G when 
G is drawn from the uniform distribution G(n, 1/2). 

Note that the probability bound 3/4 in H' e refers to the internal coin tosses 
of the algorithm. 
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Theorem 7. Set k = J 1 - 2 ^ 1 ^) where e e]0, l/2[. Sei a/so 5 = n -5^/4+€ 2 /2. 

Hypothesis (H' e ) implies that no polynomial time algorithm can distinguish 
a matrix with RIP parameter of order k at most 5 from a matrix with RIP 
parameter of order k at least n _e /4. 

Moreover, polynomial-time distinction between these two cases remains im- 
possible even for matrices of size 2n x WOn. Since 5 = o(n~ e /A), it follows that 
for matrices of this size no polynomial time algorithm can approximate the RIP 
parameter of order k within any constant factor. 



Proof. As in the proof of Theorem [6] we construct from a graph G a matrix of 
the form 

C(G) 
B„ 



C'{G) 



For B n , instead of of the deterministic construction from [7J [8] we will use 
a n x 99n random matrix given by Theorem [2j As before, we will certify 
that G does not contain a clique of size n^~ e if the hypothetical distinguishing 
algorithm A for matrices of size 2n x lOOn accepts C'(G). This will yield a 
contradiction with Hypothesis (H' e ). 

If G contains a clique of size n 1 / 2 ~ e , we saw in the proof of Theorem [5] that 
C{G) does not satisfy the RIP of order k with parameter n _e /4. By Lemma [21 
the same is true of C'{G). 

Consider now the case where G is drawn from the G(n, 1/2) distribution. 
We saw in the proof of Theorem [5] that for most G, C(G) satisfies the RIP 
of order k with parameter 6. As to B n , note that n5 2 = n e _5<E / 2 + 1 and the 
exponent e 2 — 5e/2 + 1 = (2 — e) (1/2 — e) is positive. Hence it follows from 
Theorem [2] that with probability approaching 1 as n — > +oo, B n satisfies the 
RIP of order k with parameter 5. We conclude from Lemma [2] that in this case, 
C'(G) satisfies the RIP of order k with parameter 5 for most G. □ 

The constant 100 in Theorem [7J can be replaced by any constant larger 
than 2. Note also that the hypothetical polynomial-time algorithm in this 
theorem remains deterministic: it is only the (hypothetical) algorithm for cer- 
tifiying the absence of large cliques which is randomized. It is clear, however, 
that Theorem [7] can be adapted to randomized approximation algorithms with 
one-sided error (or even with two-sided error under a suitable adaptation of 
hypothesis H' £ ). 

The following table gives a summary of our hardness results. They do not 
rule out the existence of a polynomial-time algorithm distinguishing between 
matrices with a small RIP parameter and matrices with a RIP parameter larger 
than say 0.1. Here small means as in Theorems [3] to [7] that the RIP parameter 
goes to as n — > +oo. If convergence to is not too fast then we could still 
use such a weak distinguishing algorithm for certifying most random matrices. 
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Hardness Results 


k 


(fc, 5i) vs. (fc, 5a) - hard 


Result 


Assumptions 


Dimensions (n x N) 




5! = n- 1 /*, <j 2 = n-V 6 /4 


Theorem [3] 


#1/6 


n x n 


n 0.6 


5i = n-°- 19 , <5 2 = n- 1 / 6 ^ 


Theorem [4] 


#1/6 


n x n 


n (l-2e)(l-e) 


5! = n -5'/4+ £ 2 /2 ; S 2 = n -e/ 4 


Theorem [5] 


#e 


n x n 




§i _ n -5 £ /4+e 2 /2 j ^ = n -6/ 4 


Theorem [5] 


H t 


2n x (n + n 1+e °) 


n (l-2e)(l-e) 


5l _ n -5 e /4+ e 2 /2 ; ^ = n -6/ 4 


Theorem [7] 


K 


2n x lOOn 



Table 1: We say that a matrix $ has the (M)-RIP iff (1 -<5) < ||$a;|| 2 < (l + <5) 
for every fc-sparse unit vector x. By (k,5\) vs. (/c, <52)-hard we abbreviate the 
following: no polynomial time algorithm can distinguish matrices $ that satisfy 
the (fc,<5i)-RIP from matrices that do not satisfy the (/c, f^-MP- The absolute 
constant £q > comes from [U[8]. 
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Appendix: Refuting H e for negative e 

Set k = n a where a > 1/2. In this section, we describe an algorithm which: 

(i) always outputs "yes" if G contains a clique of size k. 

(ii) Outputs "no clique" on most graphs G when G is drawn from the uniform 
distribution G(n, 1/2). 

The algorithm is as follows. 

1. Let G be the input graph and A its signed adjacency matrix. Compute 
Ai(^4), the largest eigenvalue of A. 

2. Output "yes" if Xi(A) > k — 1. Otherwise, output "no clique". 

If G contains a clique of size k, Lemma Q] shows that \i(A) > k — 1 since 
\l(A) = sup||;j.|| =1 x T Ax for any symmetric matrix. This algorithm therefore 
satisfies condition (i). On the other hand, for most G the largest eigenvalue of 
A is of order 2y/n by Theorem 2 in [15]. Since a > 1/2, it follows that most G 
satisfy the inequality Ai (A) < k — 1 and condition (ii) is satisfied as well. 
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